Evolving A Sense Of Valency

One way in which organisms learn is by feelings of pleasure and pain, which can be said to be equivalent to the reward function commonly used in computational applications of reinforcement learning. It is likely that pleasure and pain have become associated with good and bad actions through evolution, and similarly one might wish to investigate if it is possible to evolve the reward function used in computational learning. This could be useful in cases where it is not possible to design a reward function due to for example limited knowledge of the environment. This thesis investigates the evolution of a reward function, or a sense of valency, by a combination of evolution (genetic algorithms) and learning (reinforcement learning in neural networks) in a population of agents living in a grid world that contains objects that, when interacted with, affect the agents’ internal physiology. The study compares different agent architectures on a number of different environments to investigate under by which mechanisms and under which conditions a sense of valency may come about. Results indicate that the population evolves a sense of valency that increases their chances of survival. The form of the fitness assessment (continuous versus point measures) strongly affect the form of the evolved valency, as does the environment type (stable or unstable). Whereas evolution-only agents significantly outperform learning agents in a stable environment, learning agents significantly outperform evolution-only agents in an unstable environment. Furthermore, learning based on internal physiology proves to perform better than learning that is also based on environmental input. Finally, the effect of learning is determined by the ratio of the speed of acquisition of evaluative valency versus the speed of genetic assimilation of behaviour.